Papers with parameter sharing strategies
A Multi-lingual Multi-task Architecture for Low-resource Sequence Labeling (P18-1)
Copied to clipboard
| Challenge: | Existing studies have shown that multi-task learning can boost the performance of related tasks such as MT and abstractive text summarization. |
| Approach: | They propose a multi-lingual multi-task architecture to develop supervised models with a minimal amount of labeled data for sequence labeling. |
| Outcome: | The proposed architecture achieves 4.3%-50.5% absolute gains compared to mono-lingual model . the proposed model is particularly effective in low-resource settings . |
Latent Group Dropout for Multilingual and Multidomain Machine Translation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Multidomain and multilingual machine translation often rely on parameter sharing strategies, which are hardcoded in the network architecture, independent of the similarities between tasks. |
| Approach: | They propose a method to take advantage of similarities by using a latent-variable model and develop techniques to train this model end-to-end. |
| Outcome: | The proposed model improves translation performance without increasing the model size. |
A Simple and Effective Approach to Automatic Post-Editing with Transfer Learning (P19-1)
Copied to clipboard
| Challenge: | Existing APE systems generate artificial triplets of source sentences, machine translation outputs and human post-edits. |
| Approach: | They propose to use human post-edits to refine black-box machine translation (MT) models by fine-tuning pre-trained BERT models on both encoder and decoder of an APE system. |
| Outcome: | The proposed method improves on a dataset of 23K sentences on x86 GPUs. |
Parameter sharing between dependency parsers for related languages (D18-1)
Copied to clipboard
| Challenge: | Using parameter sharing between parsers of related languages can improve performance, but there is no consensus on what parameters to share. |
| Approach: | They propose a model where transition classifier parameters are shared and word and character parameters are controlled by a parameter that can be tuned on validation data. |
| Outcome: | The proposed model improves on a monolingually trained baseline. |